Voice Arcade Pi

Introduction

Our project involves designing a handheld gaming console similar to the PSP and Gameboy, with games that can be controlled using our voice. Our plan is to create two games. The first game is reminiscent of Flappy Bird, where players control a character to avoid obstacles while being able to shoot bullets to destroy them. The second game is a side-scrolling game where players control a character moving continuously forward, avoiding falling off cliffs. The console's logic is depicted in the diagram below. In the menu interface, players can toggle sound controls, select games, and view the scoreboard. When a game ends, it transitions to the game over screen, where we've hidden an Easter egg. After the game over screen, it returns to the menu.

Project Objective:

• Design two games using Pygame, implementing basic control logic, and incorporate a pause feature. Design interfaces for both the pause and game over screens.
• Use a microphone for receiving voice inputs, and integrate the game's control logic with these sound inputs.
• Design a scoreboard, allowing users to view the rankings for both games under voice control and key control.
• Set up a menu interface, enabling users to select between the two games and their control methods, as well as to check the scoreboard.
• Design an egg for the game.

Design

Game1: Flappy Fish

The first game, "Flappy Fish," is designed similarly to "Flappy Bird," featuring a character that can continuously jump to avoid obstacles. As an emergency evasion mechanism, a bullet-firing feature is added. The bullets are limited; players earn one bullet for every 300 points, starting with three bullets initially. As time (or score) increases, the difficulty level also increases.

We implement this game through the following steps:

1. Create a character capable of jumping, ensuring it does not exceed the screen's upper or lower boundaries.
2. Randomly generate moving obstacles; the game ends when the player collides with an obstacle. Also, ensure the removal of obstacles that move off-screen to clear memory.
3. Add a bullet feature to enhance gameplay and increase the player's margin for error. Bullets will destroy both themselves and any obstacles they collide with.
4. Gradually increase the speed of the moving obstacles over time to raise the game's difficulty.
5. Design the game's scoring logic (one point per frame).
6. Design the game's pause interface.

Game2: Note Jump

The second game is a side-scrolling game called "Note Jump." In this game, players move on a horizontal platform and use jumping to avoid falling off cliffs. Similar to the design approach in Game 1, we also adopted a step-by-step design process. This approach made debugging easier:

1. We implemented a character capable of upward movement, with freefall occurring upon releasing the key. Similar to "Flappy Fish," we ensured that the player cannot move out of the screen.
2. To create an infinitely long game map that allows player movement, we did not actually move the player character. Instead, we continuously generated and moved the ground to create the illusion of movement. Additionally, we ensured that ground elements exiting the screen were removed to save memory.
3. We randomly generated ground sections and introduced gaps, resembling cliffs, by making random decisions about whether to generate ground. We ensured that players could stand on the ground and fall off cliffs. The size of each generated or non-generated area remained constant, but when multiple consecutive areas lacked ground, larger cliffs were generated. During this design step, we encountered a bug. The logic for keeping the player character on top of the ground conflicted with the logic for player falling. If a collision occurred with the right side of the ground while the player was falling off a cliff, the player would be placed directly onto the ground. We resolved this issue by modifying the collision detection logic. When a collision occurred with the top of the ground, the player was placed on top of it. If a collision occurred with the left side of the ground, the player continued to fall, and player_x was set to ground_x - player_width.
4. We implemented the scoring logic, where the player's score increased by one for every frame of player movement.
5. We designed the game's pause interface.

Design Voice Control

We used the Sounddevice library to implement voice control in our project. This library can automatically detect connected audio devices and, through configuration, continuously receive and save external voice input. With this capability, we were able to easily capture audio through the microphone. We then calculated the volume of the sound using the norm function from NumPy and compared the obtained volume with a predefined threshold to determine whether an action should be executed.

One fundamental premise of voice control is that when a player encounters danger, they tend to become nervous and instinctively raise their voice. Therefore, we chose high volume levels as triggers for safety-related actions. The specific voice control logic for our games is as follows:

• In "Flappy Fish," a low volume triggers jumping, while a very high volume triggers bullet firing.
• In "Note Jump," a low volume triggers forward movement, and a slightly higher volume triggers jumping.

When designing the voice control for "Flappy Fish," we wanted the player to fire bullets when their voice exceeded a certain decibel level. However, in the initial version, shouting once would result in the firing of multiple bullets. To address this, we added logic similar to button debouncing to ensure that each loud shout only fired one bullet.

While PyAudio is a more commonly used library for audio processing, we initially attempted to use PyAudio to implement voice control. However, when we tried to run our code on a Raspberry Pi, we encountered some environment issues. During our troubleshooting efforts, we unfortunately had a significant impact on the system to the point where we couldn't even use the "startx" command to display the Raspberry Pi screen on a monitor. Fortunately, we had backups and were able to revert to a previous stable system. We decided to use the Sounddevice library as a replacement for PyAudio.

During the design process, we also encountered another issue. Initially, we wanted to use pitch to trigger avoidance actions because in times of crisis, a person's voice pitch tends to naturally rise. However, through testing, we found that this approach required a high level of precision in analyzing sound data, which our microphone did not support effectively. As a result, we opted to use higher volume levels as a substitute.

Design Scoreboard

When Voice Control is turned on, the Scoreboard displays the top five scores for each game achieved using voice control. When Voice Control is turned off, the Scoreboard shows the top five scores for each game achieved using button controls. This ensures fairness on the Scoreboard because obtaining high scores with voice control is generally more challenging than with button controls. Here's how we implemented this functionality:

1. Recording Scores: When the game is over, we read the scores from a .csv file. We then add the new score to the list, sort the list in descending order to find the top scores, and keep only the top five. Finally, we save these scores back into the .csv file.
2. Multiple .csv Files: We maintain four separate .csv files to store scores for different scenarios: Voice Control On for Flappy Fish, Voice Control Off for Flappy Fish, Voice Control On for Note Jump, and Voice Control Off for Note Jump.
3. Score Retrieval: When a player wants to view the Scoreboard, we read the corresponding .csv file and display the scores on the PiTFT screen.

Design Menu

We design the menu in the following steps:

1. Implement four options and allow players to select one of them by pressing buttons. Print the corresponding option in the terminal.
2. When a player chooses one of the options, replace the print function from step 1 with a call to the encapsulated functions to run the game or display the scoreboard. Originally, we intended to initialize and quit pygame each time we called these two functions to run the game. However, we encountered a segmentation error using this approach. Therefore, we initialized pygame only once in the menu (our main program) and quit pygame when exiting the main program. We passed the "screen" as a parameter to the called game functions.
3. Upon returning from the encapsulated game functions, based on the returned score, we designed a game over interface to display the player's score. This interface hides an easter egg that can be discovered by downloading and running our code.

Integration and Deisgn GPIO button control

1. All the functionalities mentioned above were developed and tested on a PC because it's more convenient for testing and debugging on a PC. Additionally, installing libraries or making changes directly on a Raspberry Pi can potentially impact the stability of the Raspberry Pi system. In fact, we've observed instances where other groups' systems crashed, requiring them to reinstall the system. Therefore, after implementing all the functionalities on the PC, we transferred the code to the Raspberry Pi for execution and testing.
2. During development and testing on the PC, we used keyboard input for control. However, when transitioning to the Raspberry Pi, we wanted to create a standalone, integrated gaming console without the need for an external keyboard. To achieve this, we utilized the four GPIO buttons provided by the PiTFT for control.
3. When calling different games, conflicts can arise due to GPIO configurations. Therefore, we used GPIO.remove and GPIO.cleanup to clear the configurations and then reconfigured the GPIO after calling each game.
4. Additionally, we leveraged the knowledge from lab3 to implement auto-start functionality for the game upon booting up the Raspberry Pi.

Drawings

In the initial stages of designing the game, we used simple rectangles to represent characters and objects within the game for ease of debugging. Later on, we use PiSKEL, an online pixel art design website, to design bullets, characters, and obstacles. Additionally, we used ChatGPT to generate background images, enhancing the visual aspects of the game.

Future Work

For future work, we can optimize the code structure by abstracting and encapsulating similar code into functions. Additionally, since our designer has a gaming console, we can add more games. We can also enhance the existing two games to make them more enjoyable, such as adding rewards like coins.

Result

ChatGPT Our project has successfully implemented the expected functionality. Players can select and play two games from the menu interface, with both games supporting both sound and button controls. Each game also keeps track of the top five scores for both button and sound controls. This project incorporates various concepts from previous labs, such as auto-starting the games on boot and designing Easter eggs. Overall, we are very satisfied with the design of this gaming console.

Parts List

• Raspberry Pi: provided by the lab
• Mini USB Microphont: provided by the lab

Total: $0, all parts are provided by the lab

References

PyGame
Mini USB Microphone Document
Sounddevice Library
PyAudio Library